The present invention relates to the visual assessment of a displayed digital image to yield a numerical representation of the perceived overall image quality.
Most perceptual experiments in the past have reported results in terms of ratings scales that are indefinite or unstable, e.g. in terms of adjective descriptors such as xe2x80x9cgoodxe2x80x9d or xe2x80x9cfairxe2x80x9d. This limits the value of these experiments to their immediate conclusions, and prevents subsequent application of the results to new problems. In addition, standard psychophysical test methods such as paired comparison, category sort, etc. are subject to one or more of the following problems (see Part II of C. J. Bartleson and F. Grum, xe2x80x9cOptical Radiation Measurementsxe2x80x9d, Vol. 5, Academic Press, New York, 1984). Examples of problems found in standard methods include: low precision, stimuli-induced range effects, limited quality range characterized, and difficulty of inference of perceptually relevant differences.
U.S. Pat. No. 5,400,667, teaches that a reference image series varying in one perceptual attribute and calibrated in terms of a corresponding objective measurement may be used to perform a visual assessment of the objective measurement value associated with a test sample by identifying which member of the reference image series is most similar in the appearance of the varying perceptual attribute. For example, a reference image series varying in resolution in a known fashion can be used to perform a visual assessment of the resolution of a test image. A small number of adjective descriptors (e.g. xe2x80x9cgoodxe2x80x9d or xe2x80x9cfairxe2x80x9d) are associated with the measurement scale to provide a crude perceptual characterization. For example, a resolving power of 1 cycle/mm might be considered xe2x80x9cpoorxe2x80x9d and 5 cycles/mm xe2x80x9cgoodxe2x80x9d. While the method of U.S. Pat. No. 5,400,667 can reduce the magnitude of the examples of problems mentioned above, it is still deficient in a number of respects, as set forth below
(1) It provides a measure of only a single perceptual attribute, namely, the perceptual attribute that varies within the reference image series. Practical imaging systems produce images with many combinations of perceptual attributes, so that customer satisfaction will be only partially correlated with any single perceptual attribute. What is of much greater relevance is the overall image quality. The term xe2x80x9coverall image qualityxe2x80x9d means a subjective determination by a user of an image reflecting the relative acceptability of such image to the user. For example, if a test image had xe2x80x9cgoodxe2x80x9d resolution but xe2x80x9cpoorxe2x80x9d color quality, matching the appearance to a reference image series varying in resolution according to the method of U.S. Pat. No. 5,400,667 would yield an estimate of the objective measurement value of resolving power (approximately 5 cycles/mm) and an associated adjective descriptor of xe2x80x9cgoodxe2x80x9d. However, the overall image quality of the test image would be lower than this due to the poor color quality.
(2) The adjective descriptors provide only a very coarse classification and are not quantitative in nature. The same adjective descriptors are interpreted to mean different things by different people, and the application of adjective descriptors changes over time. The differences in quality between adjacent adjective descriptors is not uniform and is larger than the differences that need to be characterized in product design.
(3) The method also does not include a provision for individual calibration of reference image series depending on their scene content. It is well known that scene content significantly affects the impact of different attributes on overall image quality. For example, the overall image quality of complex scenes with considerable fine detail is relatively less affected by the presence of a given amount of noise, as determined by objective measurement values, than is that of simpler scenes with large areas of slowly changing color (e.g. blue sky; see D. M. Zwick and D. L. Brothers, xe2x80x9cRMS Granularity: Determination of Just-Noticeable-Differencesxe2x80x9d, SMPTE 86, pp. 427-430, 1977).
(4) U.S. Pat. No. 5,400,667 discloses reference image series spanning the range from xe2x80x9cvery poorxe2x80x9d to xe2x80x9cvery goodxe2x80x9d, which may not be a good match to the range of test images to be evaluated in a particular application.
(5) The method does not identify a preferred relationship among the increments of change between the adjacent members of the reference image series, yet this affects the speed and precision of the visual assessments.
(6) U.S. Pat. No. 5,400,667 does not discuss requirements for the viewing environment in which the visual assessment is made; however, this must be carefully controlled to obtain accurate and reproducible results.
(7) The method does not include a provision for evaluation and correction of user bias due to, for example, a tendency in the visual assessment to avoid approaching end numbers in the reference image series.
It is an object of the present invention to provide a method of measuring perceived overall image quality of a digital test image in terms of a numerical scale representing overall image quality.
This object is achieved by a method for producing a numerical representation of the perceived overall image quality of a digital test image, comprising the steps of:
(a) creating a digital reference image series of a reference scene that differ from each other in at least one perceptual attribute, wherein each digital reference image has a corresponding numerical representation of its overall image quality;
(b) iteratively displaying one of the digital reference images and comparing that digital reference image with the displayed digital test image, continuing the process until the numerical representation of the digital test image may be satisfactorily inferred; and
(c) recording the numerical representation corresponding to the digital test image.
The present invention incorporates a digital reference image series that is calibrated against a standard numerical scale of overall image quality and is presented in a fashion that facilitates direct comparison of any of the digital reference images with the digital test image under matched viewing conditions. The present invention permits visual assessment of the overall image quality of the digital test image compared to the most similar digital reference images from the digital reference image series. The present invention also permits untrained personnel to perform calibrated visual assessments of overall image quality with accuracy and precision comparable to that of highly trained professionals using standards of limited availability.
Additional advantages of the present invention include:
(1) The present invention provides a measure of overall image quality rather than a single perceptual attribute, the former being more strongly correlated with customer satisfaction.
(2) The present invention produces a numerical representation of overall image quality that may be calibrated to an established scale with desirable properties such as associated physical standards, known increments of perceptual relevance, and general acceptance.
(3) The present invention provides for individual calibration of digital reference image series based on different reference scenes. This improves the accuracy of the method by accounting for the well-known variation of the effect of a perceptual attribute on overall image quality due to scene content (see D. M. Zwick and D. L. Brothers, xe2x80x9cRMS Granularity: Determination of Just-Noticeable-Differencesxe2x80x9d, SMPTE 86, pp. 427-430, 1977).
(4) The present invention permits optimization of the number of members in the digital reference image series and the increments of change between them, taking into account the range of overall image quality of the digital test images and the required precision. Proper choice of these properties minimizes the time spent by the user to produce numerical representations of a given required precision.
(5) The present invention creates a viewing environment in which important viewing factors such as viewing distance, viewing angle, illumination level, user adaptation level and flare light can be precisely matched between the displayed digital test image and the displayed digital reference image. This improves the accuracy of visual assessments by eliminating sources of bias. The present invention further permits such viewing factors to be fixed at specific prescribed values, which is important when these factors affect the appearance of perceptual attributes of either the digital test images or the digital reference images.
(6) The present invention permits correction for user and experimental biases by including within the digital test images one or more images that are identical to members of the digital reference image series either in terms of appearance or objective characteristics.
The performance of the present invention has been tested in several ways that demonstrate its effectiveness. First, the uncertainty in a single assessment has been measured and found to be two to four times smaller than standard psychophysical methods such as magnitude estimation and category sort problems (see Part II of C. J. Bartleson and F. Grum, xe2x80x9cOptical Radiation Measurementsxe2x80x9d, Vol. 5, Academic Press, New York, 1984). This uncertainty is comparable to that achieved by highly trained professionals using cumbersome standards. Second, the accuracy and precision of results obtained when the digital test image depicts a different scene than that of the digital reference series have been found to be just as good as in the case where the scenes are matched, a surprising result. This permits digital test images depicting arbitrary scenes to be evaluated without loss of performance.